Scalable And Robust Multi-Agent Reinforcement Learning